Thoughts on the Japanese Language Model

2023-02-21

I was asked the question verbally a few days ago and answered it verbally, but I drew the diagram again because I couldn't convey the image in my brain in words.

https://gyazo.com/0fd55d4e22a251d89783a76b553e1d01

For Thumbnail

Write down your thoughts on the question, "Can creating a language model specifically for the Japanese language give it an advantage over existing language models due to circumstances unique to the Japanese language?

Q: Isn't the Japanese language more expressive than English?

A: I don't think so.

Natural language has developed for the purpose of communication between homo sapiens

So "getting the message across" is most important.

The average listener's ability to understand the language does not increase due to bottlenecks in the ability to express the language.

If the "average listener's ability to understand" does not differ significantly by ethnicity or race, then the natural language used by that ethnic group will also have the same ability to express itself.

https://gyazo.com/0272f4ed04fec479f607d1746b7d4b64

(In science fiction terms, people who have lived in a harsh colony for generations might have a higher level of comprehension than the average homo sapiens, but, well, it's not much different on Earth today.)

For example, in mathematics, artificial vocabulary and syntax are used to distinguish "concepts that are used vaguely and indistinguishably in natural language.

This artificial language need not be understood by the average homo sapiens.

So it can have higher expressive power than natural language.

https://gyazo.com/e17a7ab64ff1443afc81f1eac6d8e1da

However, the ability to express yourself in areas you are not interested in is sometimes discarded.

Both English and Japanese are natural languages, which caps the ability to express oneself.

Easy compared to the ability to understand "the type of language that can only be understood by trained experts," such as technical discussions among mathematicians or the behavior of complex algorithms

Side-by-side view

https://gyazo.com/6f0ec74f72e739cdb53f7ce728d86c19

There is a difference or there isn't.

https://gyazo.com/f8b1dc87ff8d2921d8d83262d5e2e755

For example, in Japanese, the subject is naturally omitted, making it easy to discard the subject.

This, in turn, means that it is easy to overlook the subject distinction.

This means that there is a cultural difference in what is chopped up into small pieces and what is chopped up into large pieces in articulating the world.

It is not that one language is superior to the other.

https://gyazo.com/63d82ae9818edd32db83088b8b32a447

A person of one culture looks at the language of another culture and thinks, "I'm an idiot for calling B without distinguishing between B and C," but the other culture thinks likewise, "I'm an idiot for calling D without distinguishing between C and D."

There won't be much difference in the performance of the human brain, so there won't be much difference in the number of pieces inscribed in the world as a whole.

concrete example

https://gyazo.com/8eed66bb5b786b32fde95bd54ee74200

https://gyazo.com/2cea99f4304a55ae451c76b6edd1863a

I told the AI that it was an A. He said no, it's a B.

Using multiple languages, with the most finely divided language for each domain, provides a better understanding of the world than any single language.

https://gyazo.com/0a76bf10554bacb8610001409354659b

By mapping the system to the one that is easiest to operate, operating it there, and then returning to it, it is easy to do things that were difficult to do in the original system.

Better to have several different systems.

Ancient India, which developed before European ideas flowed in and was not given back to Europe much due to high translation costs.

Muslim community that had an incentive not to share knowledge due to religious opposition to Christianity.

It would be interesting to see better language models and low-cost references to literature from these cultures.

It is also worth noting that these areas are still likely to see population growth

A Japanese person asks a question in Japanese, and the AI thinks once in Arabic, then thinks in Pali based on the result, and finally answers by translating the conclusion into Japanese, saying, "This guy only understands Japanese, so it's no use.

Q: Is there value in a better language model for Japanese?

A: Yes.

This does not mean that "Japanese-only models have value."

The importance of a thicker pipeline to the "AI that thinks across languages" that will grow more and more in the future.

https://gyazo.com/0fd55d4e22a251d89783a76b553e1d01

'AI that thinks across languages' is like a newly discovered oil field, value bubbling up

Users of languages with narrower pipes do not enjoy much of the value that springs from this.

relevance

---